Detecting Classical Phase Transitions with Renyi Mutual Information 
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By developing a method to represent the Renyi entropies via a replica-trick on classical statistical 
mechanical systems, we introduce a procedure to calculate the Renyi Mutual Information in any 
Monte Carlo simulation. Through simulations on several classical models, we demonstrate that 
the Renyi Mutual Information can detect finite-temperature critical points, and even identify their 
universality class, without knowledge of an order parameter or other thermodynamic estimators. 
Remarkably, in addition to critical points mediated by symmetry breaking, the Renyi Mutual Infor- 
mation is able to detect topological vortex-unbinding transitions, as we explicitly demonstrate on 
simulations of the XY model. 



Introduction - The universality and importance of the 
concept of information is exploited widely in mathemat- 
ics and the physical sciences. Information sets the funda- 
mental limits in communication (or uncertainty) , regard- 
less of system, technology, or physical material. Shannon 
was the first to quantify information using the concept of 
entropy - a quantity that has its roots in thermodynam- 
ics [lj. It is therefore not surprising that deep ties exist 
between the measurement of thermodynamic quantities 
and concepts associated with information theory. 

There is already a rich cross-fertilization between ideas 
in condensed matter physics and the information sci- 
ences. Most recently, information measures have been 
used to quantify "hidden" correlations in materials - 
exchanges of information that can occur between two 
parts of a system that are not manifest in traditional 
condensed-matter estimators (such as correlation func- 
tions) 0. For example, in spin liquid phases, correlation 
functions can rapidly decay as a function of spatial sep- 
aration; however, due to constraints, hidden correlations 
exist across vast distances of the sample Q . These can be 
manifest in entropy quantities measuring the amount of 
communication between two regions of the sample - re- 
sulting in a practically useful estimator for, among other 
things, topological order in condensed matter systems 

m 

Phase transitions offer another testing ground for the 
use of information quantities in condensed matter sys- 
tems. Critical points arc associated with a diverging 
correlation length, suggesting the existence of long-range 
channels for information transfer. However, it is not ob- 
vious that useful, measurable, quantities associated with 
this information can be exploited to tell us anything of 
practical importance about these phase transitions. 

In this paper, we examine the Renyi Mutual Informa- 
tion (RMI), a measure that quantifies the amount of in- 
formation contained in some region of a classical statis- 
tical mechanical system, about the rest of the system. 
Numerical measurements of classical mutual information 
@; 13 typically calculate the reduced density matrix ex- 



plicitly, or use symmetries to reduce its size, which give 
the von Neumann entropy directly but are computation- 
ally quite expensive. The RMI on the other hand is 
easily measured in standard Monte Carlo routines, re- 
quiring calculation via a repUca trick |,0 rather than a 
reduced density matrix, making it immediately amenable 
to measurement on a vast number of models of interest 
to condensed matter, biophysics, and physical chemistry. 
We show that this RMI can be used in a practical way 
to identify finite-temperature phase transitions through 
finite-size scaling analysis on lattices of different sizes, 
without knowledge of an order parameter or any other 
thermodynamic quantity. The standard two-dimensional 
Ising model is used as a test case, demonstrating uni- 
versality when results are compared to vastly different 
(even quantum mechanical (loj ) models that exhibit the 
same universality class. Finally, we establish the abil- 
ity of the method to detect the Bcrczinskii-Kostcrlitz- 
Thouless transition, without relying on knowledge of any 
thermodynamic estimator such as the spin stiffness. This 
suggests the power of the RMI for detecting hidden tran- 
sitions in a variety of other statistical mechanical models 
in the future. 

Information, entropy, and the replica trick - Given a 
random variable X one can quantify its associated un- 
certainty, or equivalently, the amount of information one 
is missing by not knowing the state of X. There are var- 
ious ways this can be measured, one of which is famously 
called entropy, embodied for example in the generalized 
Renyi entropies 11| . 
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where pi is the probability of outcome X = i. Taking the 
limit a — > 1, one recovers Shannon's familiar entropy, 
Si = — X)iPi m (Pi); which can be related to the ther- 
modynamic entropy of a statistical mechanical system 
S = In fl, where f2 is the number of microstates, assum- 
ing all occur with equal probability. This relationship 
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is often exploited in the study of real physical systems 
through microscopic statistical mechanical models. 

In such systems, one may also ask how much knowledge 
of a subsystem (call it A) is possible, assuming complete 
knowledge of another subsystem B. Although seemingly 
unmotivated in this language, correlation functions are a 
common example that incompletely quantify this knowl- 
edge. For the purposes of this paper, let us define a spa- 
tial subregion A as the complement of B, so that A UB is 
the complete system; we could say that pi A is the prob- 
ability of state %a occurring in region A. The state %a 
could be any classical (or indeed quantum) state: we will 
restrict our focus mainly to classical spin systems. 

From Boltzmann, the probability of a given state in 
region A and region B for a classical spin system can be 
written 

e -PE(i Al i B ) 

where E(iA, is) is the energy of a system in states %a in 
region A and is in region B, and Z = t e -pE(i A ,i B ) 
is the partition function. Eq. ([2]) is the quantity required 
to measure the entropy of the entire system - regions 
A and B. However, to obtain only the probability of a 
certain state in region A we can instead trace over all 
possible states in region B, 

e -/3E(i A ,i B ) 



Summing over the quantities in Eq. ^ or ([3]) raised to 
the power a, the Renyi entropies can be obtained in a 
straightforward way. In equilibrium statistical mechani- 
cal systems, physical estimators based on sums such as 
this are commonly calculated using importance sampling 
techniques, i.e. Markov Chain Monte Carlo. 

In this paper we restrict our discussion to the second 
Renyi entropy which requires sums of the probabilities 
squared in Eq. and ©. For example, to get the en- 
tropy of region A we use, 




leading to a Renyi entropy of 

S 2 (A) = - In r^SSe-W^tW.)) , 
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(5) 

which can be sampled via a Monte Carlo simulation using 
the procedure described in the next section. 

Note that, while the Renyi entropies tell us about un- 
certainty in the full system or part of the system, they do 
not reveal the correlations or information between two re- 
gions of a system. For this we must introduce the Renyi 




FIG. 1: (color online). The replica trick for calculating S2(A) 
in a one-dimensional systems with 10 spins. In region A (the 
left 5 spins), configurations are constrained to always be iden- 
tical. In region B, configurations are sampled independently 
in the two replicas. 

Mutual Information (RMI), 

I a (A;B) = S a (A)+S a {B)-S a (AuB). (6) 

This measure defines in a precise way the information 
that a full knowledge of B gives us about A, or vice versa, 
since it is symmetric in the regions A and B. One can see 
that the three constituents of I a (A; B) are the so-called 
generalized Renyi Entropies defined above, including the 
entropy of region A, of region B, and of the full system 
AUB. 

Monte Carlo Algorithm - For any physical system, the 
calculation of the Renyi entropy of a region, A, can be 
accomplished by sampling the quantity in Eq. (j4]) in a 
Monte Carlo simulation with a modified simulation ge- 
ometry. Identifying the two individual sums in Eq. ((4]) 
with two independent replicas of the simulation imme- 
diately leads to a practical method for measurement of 
S*2 in Eq. ([5]), where the states of the subsystem B are 
sampled independently via two replicas (the sums over %b 
and js), while the spin state in region A is constrained 
to be identical between these two replicas (the sum over 

A schematic representation of a modified simulation is 
shown in Fig. [TJ The constraint of identical region ^4s 
demands that, to be accepted, a spin update in region A 
must update spins in both replicas. For a spin in region 
A away from the boundary, because each replica of the 
system is at a temperature 1//3, the acceptance rate of 
such a move has the form e~ 2/3AE , where AE is the cost 
of the update in just one of the layers. This factor of 2 
may be interpreted as moves in region A away from the 
boundary occurring at half the temperature as a move 
away from the boundary in region B 

The above method lets us generate states according to 
the probability in Eq. 2J but an additional step is needed 
to calculate the Renyi entanglement entropy. One ap- 
proach, used previously [12], is to integrate the energy 
estimator, starting from infinite temperature, down to 
the temperature of interest, obtaining the Renyi entropy 
from both a replicated simulation and an unreplicated 
simulation. This necessitates a schedule of simulations 
over a large range of temperatures to gather enough de- 
tailed knowledge of the energy to integrate it. Alterna- 
tively, one may use methods able to directly extract the 
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FIG. 2: (color online). Left: The RMI per boundary length 
(12/i) as a function of temperature for the Ising model. 
Dashed lines indicate T c and 2T C . Upper right: the first 
derivative of the RMI for each system size as a function of 
temperature, with a fit to a logarithmic divergence shown in 
Eq. p]l. Lower right: the temperature of the lower crossings 
of the RMI for sizes L and 2L as a function of l/L. Dashed 
line indicates T c . Finite size scaling gives us T c — 2.2683(17). 



FIG. 3: (color online). The RMI per boundary length {h/l) 
as a function of temperature for the classical XY model, 
dashed lines indicate Tbkt and 2Tbkt- Inset: The same 
plot for the quantum spin- 1/2 XY model generated using a 
Wang-Landau technique [l^]. Note that classically we reach 
much larger systems (only smallest and largest sizes are la- 
beled). 



ratio of partition functions at a given temperature. This 
is done by constructing moves that are able to connect 
simulations with different region sizes in a way that satis- 
fies detailed balance (for a quantum approach, see 13} ). 
By calculating the acceptance rate of moves connecting 
partition functions of differing region sizes, we can calcu- 
late the ratio of partition functions which can be related 
to the trace of higher powers of the density matrix ■ 

Results on models - Using conventional Monte Carlo 
simulations of several models, we demonstrate the use of 
the second RMI to detect finite-temperature phase tran- 
sitions. In the following, we use hiA; B) where A and B 
are complementary regions, each defined as an L x L/2 
cylinder embedded in the L x L torus. 

The first model we examine is the classical Ising model 
on a two-dimensional square lattice 
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where Sf = ±1/2. There exists a transition to an or- 
dered phase at a temperature T c /J = 2/ log(l + y/2) ~ 
2.269 0], which offers us the simplest test for the RMI. 
Figure [2] illustrates h {A; B)/£ where £ is the length of 
the boundary between the regions (in all cases hereafter, 
£ = 2L). Close inspection indicates approximate cross- 
ings of the Ii{A; B)/£ curves at T c and 2T C . Examining 
the crossings as we move to larger system sizes we see 
that they extrapolate to the transition temperature as a 
function of l/L, as illustrated in the inset of Figure [2j 

Remarkably, this behavior mimics the crossing seen 
previously in the analogous RMI quantity based on 
the entanglement entropies in a quantum spin-1/2 XXZ 



model [12L Il5| . That model also realizes a finite- 
temperature critical point, in the 2D Ising universality 
class (however at a different T c , which is non- universal) . 
There, it has been determined that the crossings were a 
manifestation of criticality, with scaling form, 

I 2 (A;B) = [ Cl (t)+ tlogt]-£ + c 2 (t) + 0(l/£), (8) 



with t = \T - T c \/T c [lOj. It is important to note that 
c±(t) can be polynomial in t, where the strict "area law" 
at t = is caused entirely by its constant piece, while 
divergences in derivatives of I2 {A; B) are caused only by 
the t\ogt contribution. This diverging term is known 
from the tlogt divergence in the boundary free energy 
as t — > for the square lattice Ising model with a field 
applied to the boundary on an infinite half plane 16|, 1 1 71 ] ; 
we can thus use it as a test of universality in this model. 
In Figure [2j we plot the first derivative with respect to 
t of the RMI and see the formation of a logt singular- 
ity, in agreement with this scaling form. There, only the 
largest system studied has good agreement to this form - 
illustrating an advantage of using classical Monte Carlo 
over quantum Monte Carlo (QMC) techniques, the for- 
mer being amenable to simulations on significantly larger 
system sizes. 

We turn now to Monte Carlo simulations of the classi- 
cal 2D XY model on a square lattice, 
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This is a model with continuous spin variable < 
9i < 2tt that undergoes a Berezinskii-Kosterlitz-Thouless 
(BKT) [18( transition from a phase with free vortices to 



one with bound vortex-antivortex pairs. Numerically, the 
detection of this phase transition is much more subtle 
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than the Ising phase transition, since the BKT transi- 
tion has no local order parameter and is typically iden- 
tified by the change from an exponential decay in corre- 
lations above the transition to a power law below fljj ]. 
Because the BKT transition exhibits this "essential sin- 
gularity" , standard scaling theory does not work, making 
its detection via typical thermodynamic estimators (such 
as the specific heat) impossible. In 2D, this is circum- 
vented by measuring the energy cost of uniformly twist- 
ing the spins in any lattice direction (the "spin stiffness" ) , 
and making use of a special universal "jump condition" , 
Tbkt = tt/9s/2, due to Nelson and Kosterlitz By 
examining the spin stiffness in this way, the transition 
temperature has been found to be Tbkt = 0.89294(8) 
[iol ]. However, one may wonder if the RMI can detect 
this phase transition without any need for specialized 
measurements like the above. 

To address this, we examine the RMI for the classical 
XY model, shown in Figured One can clearly see the de- 
velopment of crossings in the quantity l^ji near the value 
of Tbkt and 2Tbkt- This is a strong indication that some 
sort of critical scaling is coming into play near the BKT 
transition. To examine this concept further, in the inset 
we illustrate the RMI for a completely different model, 
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the quantum XY model H = —fy\ l{ - s . 

— v'J } \ * J * 3 j 

computed using Stochastic Scries Expansion QMC where 
the quantum RMI is generated using a broad histogram 
approach [l5j. In this case, similar crossings appear at 
the (non-universal) Tbkt ~ 0.343, giving strong evidence 
in support of the universality of this result that the RMI 
can detect the BTK transition. 

However, unlike the relatively "clean" crossing of the 
2D Ising critical point, these crossings have a larger finite- 
size scaling component. Figure U contains a detailed 
finite-size scaling analysis for the model in Eq. This 
shows fits to two finite-size scaling forms which are de- 
rived by setting the correlation length near the critical 
temperature, £ = e~ b ^(l + 0{t)) 0|, equal to the lin- 
ear system size L/Lq with t = (T — Tbkt) /Tbkt- Then 
by relating the temperature T with the finite-size transi- 
tion temperature T(L) gives: 



Tbkt(L) — Tbkt(oo) 



1 



log 2 (L/Lo) log 5 (T/To) 



(10) 



Note, typical finite-size scaling analyses set the coefficient 
c equal to zero, effectively ignoring subleading 0(t) cor- 
rections to the correlation length. Using this finite-size 
scaling from, the data for the crossing of I2/I convinc- 
ingly approaches Tbkt in the limit of L — > 00. 

Discussion - The Renyi Mutual Information (RMI) is 
a quantity able to detect all correlations in a physical sys- 
tem, even those missed by traditional connected correla- 
tion functions. We have introduced a practical method 
to calculate the RMI using a modification of standard 
Monte Carlo techniques for classical statistical mechani- 
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FIG. 4: (color online). The crossing of the RMI for the 
classical XY model between sizes L and 2L as a function of 
l/log(L) 2 . Fit 1 fits the points to Eq. (|10[1 assuming c — 
and using the largest six systems while Fit 2 uses the full equa- 
tion and all of the data. They give Tbkt estimate of 0.912(4) 
and 0.899(9), respectively. Inset: close-up of the lower cross- 
ing (only smallest and largest sizes are labeled). The arrows 
indicate the eight crossing values used in the main figure. 



cal systems. We demonstrated that the RMI associated 
with the second Renyi entropy, I2 , is able to identify both 
conventional critical points, as well as continuous transi- 
tions where standard scaling theory breaks down, namely 
the BKT transition. A straight-forward finite-size scaling 
analysis of I2 is sufficient to identify each phase transi- 
tion, without knowledge of an order parameter, broken 
symmetry, or critical theory. 

The ease of implementation of the RMI measurement 
into any standard Monte Carlo routine could stimulate 
adoption to simulation studies of many fields of the phys- 
ical sciences and beyond. Based on its ability to detect 
both Ising and BKT transitions, it would be interest- 
ing to implement the RMI measurement in simulations 
of generalized XY models where conventional estimators 
fail to identify such phase transitions (22[. The RMI 
will likely find great utility in other classes of classical 



models, such as loop [23|, |24| or dimer models [25|, |2 



where universal properties of exotic criticality may be 
manifested [27|- Also particularly pressing is the ques- 
tion of whether RMI measurements can detect uncon- 
ventional transitions in disordered systems, such as the 
glass [2^, [29[ or jamming transition [3(| • Finally, the 
ubiquity of the Monte Carlo method in such far-reaching 
fields such as humanities or finance may precipitate the 
use of RMI in a host of unforeseen applications. In par- 
ticular, for systems where analyzing mutual information 
in pre-existing data sets has proven illuminating, such as 
detecting transitions linked to financial market crashes 
31j . RMI may provide a predictive tool when used in 
conjunction with Monte Carlo simulations of models. 
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